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We present relaxed notions of simulation and bisimulation on Probabilistic Automata (PA), that al- 
low some error £. When £ = we retrieve the usual notions of bisimulation and simulation on PAs. 
We give logical characterisations of these notions by choosing suitable logics which differ from the 
elementary ones, Jz? and jSf -1 , by the modal operator. Using flow networks, we show how to com- 
pute the relations in PTIME. This allows the definition of an efficiently computable non-discounted 
distance between the states of a PA. A natural modification of this distance is introduced, to obtain a 
discounted distance, which weakens the influence of long term transitions. We compare our notions 
of distance to others previously defined and illustrate our approach on various examples. We also 
show that our distance is not expansive with respect to process algebra operators. 

Although jSf t -1 ) is a suitable logic to characterise £-(bi)simulation on deterministic PAs, it is 
not for general PAs; interestingly, we prove that it does characterise weaker notions, called a priori 
e-(bi)simulation, which we prove to be NP-difficult to decide. 
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1 Introduction 

Preorders and equivalence notions between processes are central to concurrency theory One wants to 
compare terms of a process algebra for proving an axiomatisation sound, to compare processes to some 
abstractions of them, etc. For non-probabilistic processes, notions of bisimulation and simulation are 
widely acknowledged, with, of course, many variations. In the study of probabilistic systems it has been 
observed [17] that the comparison between processes should not be based on notions that rely strongly 
on exact numbers, as do the known notions of bisimulation and simulation for probabilistic systems. 
The most important reason is that the stochastic information in probabilistic processes often comes from 
observations, or from theoretical estimations. Hence a slight difference in the probabilities between two 
processes should be treated differently from important ones and certainly not be simply tagged as non 
equivalence. In this context, notions of approximate equivalence or distance are more useful. Distances 
have been defined for probabilistic processes IH31 0) and some have tried to estimate bisimulation with 
a certain degree of confidence |[T5l . Relaxing the definition of simulation and bisimulation is another 
avenue, which we follow. 

We first extend previous work on deterministic processes lfT2l to their non deterministic version, 
Probabilistic Automata (PA) ll22ll . We present relaxed notions of simulation and bisimulation on them 
with respect to some accuracy e. When e = we retrieve the usual notions of bisimulation and simula- 
tion on PAs. Our notions rely on a definition of e-lifting of relations, which happens to be equivalent to 
the one presented in (23). However, in this paper, the authors present different notions of e-simulations 
which consider distributions on the set executions, whereas our relations are always between the states 
of the systems, and our purpose is different. We give logical characterisations of these notions: a state 
£-simulates another state if and only if it e-satisfies every formula that the other one (exactly) satisfies; 
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similarly for £-bisimulation. The extension of previous work comprises also the definition of an effi- 
ciently computable non-discounted distance: two states are at distance less than or equal to £ if they 
are £-bisimilar. Using flow networks, we show how to compute in PTIME our relaxed relations of 
(bi)simulation which helps to also compute efficiently the distance. 

The nature of non determinism leads to new challenges and concepts. It is not suprising that the 
logics that we prove to characterise £-bisimulation and £-simulation differ from the elementary ones, 5£ 
and jS? -1 . Although Jzf'^ is a suitable logic to characterise £-(bi)simulation on deterministic PAs, it is 
not for general PAs; interestingly, we define weaker notions that it does characterise on PAs, called a 
priori e-(bi)simulation. We also prove that a priori O-simulation is NP-difficult to decide, contrarily to 
£-bi/simulation. 

We propose a natural modification of our basic distance in order to discount the influence of long term 
transitions. We illustrate the difference between the values of the two distances on various examples of 
two-dimensional grids. Both (pseudo-)distances are different from the ones defined in the past iTTTl [T] 
[15] |U, in that differences along paths are not accumulated, even in the discounted one. The other known 
distances all accumulate differences through paths, and most of them discount the future. Those that 
do not discount the future are intractable: it has recently been proven decidable (3), but with double 
exponential complexity. Our distance is determined with a polynomial algorithm. 

Finally, we prove that our distances are not expansive with respect to process algebras operators, 
such as parallel composition and non-deterministic choice. 

2 Probabilistic Automata and e-relations 

In this section we give the definitions of our models and the relaxed relations that we study. Probabilistic 
Automata are labelled transition systems where transitions are from states to distributions and that in- 
volve non determinism. We generalize slightly the standard model, allowing sub-distributions instead of 
distributions, to model non responsiveness of the system and to make simulation a richer notion. Given 
a countable set S, we write Sub(»S) for the set of sub-distributions on S: the total probability out of a sub- 
distribution may be less than one. Given a relation R on S x S and X C S,R(X) = {y G S\3x GX s.t.xRy}. 
A set X is R-closed if R(X) C X. 

Definition 1 (PA |22|) A probabilistic automaton, or PA, is a tuple 5? = (S,Act, $) where S is a denu- 
merable state space, Act is a finite set of actions, and 3> C S x Act x Sub{S) is the transition relation. S? 
is finitely branching if for all s G S and a G Act, {/J. G Sub(S) | (s,a,n) G ?3>)} is finite; if it is a singleton 
or empty, we say that 5? is deterministic. The disjoint union of PAs 5f\ , ..., 5^i is the PA W,- e [i ; M.^ whose 
states are the disjoint union of the S, and transitions carry through. 



Closely related models restrict states to be either probabilistic or non deterministic 111811 . Generalizations 
to uncountable state spaces have also been studied GJ[9). We sometimes mark a state (or a distribution) 
as initial. We write s — > pt for a transition (s,a,/j.) G 0. 

An example of PA is given in Fig. [T] an arrow labelled with action I and value r represents an l- 
transition of probability r; in picture representations, we omit the distributions that are concentrated in 
one point, as can be seen for the transitions from s to so and from t to itself. In contrast, state s has an 
a-transition to distribution /i v giving s three possible successors for this a-transition. 

In previous work [12], we relaxed the classical notion of simulation between deterministic PAs to 
e-simulation. We now generalize this approach to the context of PAs. 
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Figure 1 : s -< i t and His but s <ft\t 



Definition 2 (£-bi/simulation) Let 5? = (S,Act,@) be a PA, and e > 0. A relation R C S x S is an 
e-simulation on 5? if whenever sRt, ifs^-il, then there exists a transition t — > v such that iiJ£ e (R) v, 
where 

ii^ e {R)v iff for all EQSwe have /x (E) <v(R(E))+e . 

IfR is symmetric, it is an e-bisimulation. Two states s and s' ofPAs S?\ and S?i are e-similar (resp. e- 
hisimilar), written s -< e s 1 (resp. s ~ e s'), if there is some e-simulation (resp. e-bisimulation) that relates 
them in Sf\ l±l ^ 

We may omit e in the notation when e = 0, as it yields the classical notions. 

Example 1 In the PA of Fig. U\s ~<\_ t and t -<\ s. This is witnessed by the relations 

{(s,t),(x,x),(x,t 2 ),(y,y),{y,t3)} U {(>,•,?,•) | i = 0, 1,2,3}, 

{(t,s), {x,x),(x,S2),(y,y),(y,s 3 )}u{{ti,Si) \ i = 0, 1,2,3}. 

However, s and t are not related for any e < g. Notice that we have s ^ i t. Indeed, x is ^-bisimilar to no 
state but itself, but /J. s ({x}) = „-, which is strictly greater than pL t ({x}) + g = |. 

This example shows that two-way e-simulation is not e-bisimulation, even for deterministic PAs. A 
deterministic example is obtained by removing a-loops in Fig. [T] 
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Proposition 1 e-bisimulation is different from two-way e-simulation. 

As for the classical case, we define the largest relations as greatest fixed points: F e : 2 s 
defined as follows W C S x S, Vs,t e 5, let (s,t) G F e {R) iff Vs 4 ju, 3t A v | ix^ e (R)v. Similarly, 
G e . 2 sx5 _j. 2 sx5 is defined as (j,/) G G e (tf) iff (>,*), (M) e F e (R). We then define -<2 heratively as 
-4= S x 5 and for all n, -< n e +i = F e (-< n ). As well, let ~°= S x 5 and for all n, ~g +1 = G e (~ M ). 

Theorem 1 J^ 7 being finitely branching, -< £ and ~ £ are ?/je greatest fixpoints ofF e and G e , respectively. 



In other words, -< e = H«gN ~^% an d 



n« 



2.1 Lifting of relations and flow networks. 

The lifting of a relation is a standard construction that transfers a relation 7? on states to a relation ££ (R) 
on sub-distributions over states. Contrarily to the way we formulate Def.|2j the usual definition of liftings 
is rather in terms of the existence of a weight function [22]. We show the equivalence of the definitions. 
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Definition 3 (£-weight functions) Let e > 0, and H,v £ Sub(S). An £-weight function for (jU, v) with 
respect to R is a function 8 : S x S — > [0; 1] such that: 

• If8(s,t) > then (s,t) £ R. 

• For all s,t £ S, Ey e s5(j,j') < jJ.(s) andY, s 'es^( s '^) < v (0- 

• L^e5 5(j,/)>M(5)-e. 

Before stating the equivalence between our formulation of Jz? e (R) and the one with weight functions, 
we recall the notion of flow network, since it provides a convenient alternative definition for applica- 
tions 0. 

A network is a tuple J/ = (V,E,±,T,c) where (V,E) is a finite directed graph in which every edge 
(u,v) £ E has a non-negative, real-valued capacity c(u,v). If (u,v) E we assume c(u,v) = 0. We 
distinguish two vertices: a source _L and a sink T. For v G V let in{v) be the set of incoming edges to 
node v, and out(v) the set of outgoing edges from node v. A flow function is a real function / : V x V — > R 
with the two following properties for all nodes m and v: 

• Capacity constraints: < f(u,v) < c(u,v). The flow along an edge cannot exceed its capacity. 

• Flow conservation: for each node v £ V — {_L,T}, we have £ e€i - n ( v ) /(e) = LeeoM?(v)/( e )- 
The flow &(f) of / is given by J*"(/) = Ze&ut(±)f( e )-Le^in(T)f(.e). 

Definition 4 (The network =yT(jU, V,R)) L<?? 5 fee a ./zm'te «tf, R C S x S, and fX, V £ Sub(S). Let S' = 
\t'\t £ S}, where t' are pairwise distinct "new" states (i.e. t' S). Let _L and T be two distinct new 
elements not contained in SL) S' '. The network .jV (jJ-,V ,R) = (V,2?,_l_, T,c) is defined as follows: 

• V = 1 SU5 / U{_L,T}. 

• E = {(s,t')\(s,t) £ R}U{(±,s)\s £ S}U{(t',T)\t £ S}. 

• The capacity function c is given by: c(_L,s) = n(s), c(t, T) = v(t), and c(s,t) = I for all s,t £ S. 

The following proposition gives various characterizations of the simulation relation. 

Proposition 2 Let S be a finite set, R C S x S, and /I, V £ Sub(S). The following properties are equiva- 
lent: 

1. n^ e (R)v. 

2. The maximal flow in J^{}X ,V,R) is greater than or equal to /I (S) — £. 

3. There exists an £-weight function for (jU, v) with respect to R. 

4. For all R-closed set E QS, we have n (E) < v(E) + e. 

The equivalence with 4 applies only if the domain and image ofR are considered disjoint. 

Proof. 1 44> 2 is Theorem 7 lfT2l . 2 44> 3 is a slight generalization of a result of 0, in which e = 0. 
1 ^> 4 is always true and 4 =^> 1 is straightforward if the domain and image ofR are disjoint. □ 

The condition on the fourth statement may look restrictive, but it is quite natural. By taking two 
copies of the state space and relating a state to the copies of the states that R relates it to, one obtains 
a relation that satisfies the condition, yet representing the same relation as R. This allows to make a 
distinction between states that are simulated from states that are viewed as simulating. 
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3 Logic for £ -simulation and e-bisimulation 

3.1 The logic Jzf and its corresponding notion of simulation 

In the context of deterministic PAs, £-bi/simulation are characterized [12J by the simple logic Jzf ^ |[T9ll . 
using a relaxed semantics "up to e". 

Definition 5 (The logics Jzf and Jzf^) 77je syntax of ££^ is as follows: 

S^ ::=T | -.0| 0i A0 2 | 01 V0 2 | (a)s0 w/We <5 G Qn [0;1]. 

We wn'fe J£ for the logic without negation. Given a PA with components (S,Act,@), the relaxed 
e-semantics |= e is defined by structural induction on the formulas. 

•\/s G S, S |= e T • S \= e -i0 z/f 5 ^_ e • J |= e 01 A 02 z/f 5 |= e 01 arac? S |= g 02/ 

• 5 |= e (a)g iff there exists a transition s — > p s.t. /x([[0]] e ) > 8 — £, 

where [0] e ={s£ S\s |= g 0}, and the semantics o/V is similar to the one of A. Given s G S, we write 
J^~ e (s) (resp. J^ e '^(s))for the set of formulas in Jzf (resp. in Jzf -1 ) e-satisfied by s. 

As for deterministic PAs, the logic is less expressive with the relaxed semantics [12] than with the 
standard one. Indeed, for each G Jzf" ', we can construct an associated formula e G Jzf^ such that 
Me = [0elo. Here is how this is done: T e = T; (0i A0 2 ) e = (0i)e A(0 2 ) e ; ((«)s0)e = («)s-e0e; 
(-.0)e = ->(0_ £ ). Here we use the fact that (a)x is still a valid formula, even if A < or A > 1, which 
gives in turn that (0 e )-g = 0- Clearly the transformation is additive, as (0 e ) e ' = e+e '- 

Example 2 If<(> = (a), 5 (^(a), 2 T), then e = (a).5_ e (->(a).2n 



he 



The relaxed logic being less expressive is not an issue because we use the new semantics to simplify 
the formulations of the logical characterisations. It implies that model checking of formulas with the 
relaxed semantics can be done using the same technique as for the usual semantics. 

The logics _Sf and jSf -1 induce £-simulation and £-bisimulation relations: 

Definition 6 (Logical £ -simulation.) Let e G [0; 1], s,t G S. We say that t Jzf e -simulates s, written s ~<f t, 
if for all formula G Jzf, s |= implies t \= e 0. States t and s are said £-logically equivalent, written 
s ~g t,ifs<£ t andt -<jf s. 

The following theorem says that Jzf characterizes £-simulation on deterministic PAs. As Ex. [TJillus- 
trates, £-bisimulation is different from two-way £-simulation when £ > 0, and hence we need negation 
to characterize £-bisimulation. 

Theorem 2 ([12]) For deterministic PAs we have -< e = -<f and ~ e = ~e 

As can be expected, the logics Jzf and Jzf ^ are not strong enough to characterize £-bi/simulation for 
PAs. In the next section, we will present a stronger logic that will characterize these notions. Never- 
theless, we can present the notions that correspond to -<f and ~jf . The name "a-priori" [1] comes 
from the order of the quantifiers in the definition: sets E in the following definition are chosen before the 
matching transition, which contrasts with Def . [2] 

Definition 7 (A priori £-simulation and bisimulation) Let ,5? = (S,Act, <$) be a PA. A relation R C 
S x S is an a priori £-simulation on S^ iff for all s,t G S such that sRt, for all s — >• p and for all E Q S, 
there exists a transition t — )■ V such that p{E) < v(R(E)) + £. If R is symmetric, then it is an a priori 
£-bisimulation. We write ^f'° and ~g™ c for the largest relations of a priori ^-simulation and e-bisimula- 
tion. 
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Before proving that this relation is characterized by Jzf, we introduce some notation. We define 
iteratively in the same way as ~< n e , using F pno : 2 SxS — > 2 SxS , which is defined as follows: V7? C 
5 x S, We S, let (s,t) G F prio (R) iff Vs A ju, V£a, 3? 4 v 5.?. /i(E) < V (/?(£)) + £. As for 



pno,n 



-<e= ClneN ^e> we can show that we have -< 



pno _ 



H ,pno,n 

I IneN ^e 



The depth of a formula is the maximal 



number of imbrications of (a)§ operators. We write & n for the set of formulas of ££ of depth at most n. 
Given n G N, given s€S, J^ 1 (s) is the set of formulas G Jz? of depth at most « such that s |= e and 
& e (s) is the set of formulas G if such that s \= e 0. We define ^"(s) = ^(j) and ^"(j) = &o(s). 
The next theorem proves the logical characterization of -<g"° and 

Theorem 3 Let S" = (S,Act, Si) be a PA, and let s,t G 5. Then: 



pno 



1. S^ 



prw,n 



t iff^ n (s) C &%{t)foralln> 



2. For a// « > 0, for all u,v G S, there exists <j) u G ^ n such that v |= e (j> u iffu~< 



pno.n 



CH 



pno 






3. -< t _ . 

4, ~ £ c ~£ no = ~^f owd ?/ze inclusion is strict. 

This proof generalizes to the context of countable state space systems, using a method close to the 
method used in ifTZI to extend logic characterizations to denumerable state spaces. 

Proof. Inclusions are straightforward. The proof of the strictness of inclusion is given by the 
example following the proof. The structure of the proof is similar to the ones of lfT9l and IT221 for the 
logical characterisation of simulation but we adapt them to systems with non determinism. The third 
point is a corollary of the first one. The fourth point is not more difficult and is the same kind of 
translation as the proof of lfT9l . We sketch the proof of the first two points, concentrating on the "<^" 
direction. The two points are proven simultaneously by induction on n. The base case follows trivially 
from the definitions. Assume that the claims are true for n. We now prove 2 for n + 1. Fix u G S. We 
define ty u G ^ n +\ as follows. Let v G S such that u ^f" '" v; by induction ^ n (u) % J^" (v), that is, there 
exists a formula $( U)V ) G ^ n , such that u |= ty( u ,v) an d v V=e 0(«, v )- The formula M := /\ v jt rU >^ u 0(«,v) is m 
J^"„+i because 5 is finite. Now, u |= <j> u , and any v such that u ^P no ' n+ v verifies u -/{ p e no,n v and hence 
v y=e <t>u- Since u ^P noM w implies w |= ty u by induction, we get the result; hence 2 is proven for n + 1. 
As for 1, suppose J^"" +1 (s) C &jf +1 (t). Let s — > jJ. be a transition from s, and let £CS. We are looking 
for a transition t — > v such that v(-< p e no ' n E) > n(E) — e. Let p = fi(E). We construct a formula <pE such 
that for all u G S, u \= <pE iff u G £. By the second claim, we just have to consider <j>e = V«e£ 0m- Now, 
s |= (a) p (j)E- Since (a) p (j)E S ^" +1 , we must have f |= g {a) p §E- Since -<§"°'" [0 £ ] C [</>]£ e , we get that 
there exists a transition ? — > v from ? such that v(-< p e no,n E) > fl(E) — £, which proves the result. □ 



Example 3 In the following PA (where the bj's are different labels), we have s -<P rw t and s -fit. 
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The transitions has been chosen such that \i(t\) = \x(s\), Vi{ti) = V-fa), Vi(?3) = ju(j3), Vi({fi}U 
{t 2 })=H({si}U{s 2 }), V 2 ({ti}U {t 3 }) = n({ Sl }U {s 3 }), v 3 ({t 2 }U {t 3 }) = n({s 2 }U {s 3 }). Then it is 
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easy to see that Sj is simulated by tj for all i = 1,2,3. Moreover, the last set of equalities shows that 
s ~<prio t- However, we do not have s -< t. Indeed for all transitions t — > V (combined or not) from t, we 
can find a set E C U,- e {i 2.3} {s{,ti} containing Sj if and only if it contains tj, and such that *l(E) > v(E). 

Remark 1 By Theorem\3\item 3, for the PA of Ex. \l\ we have s -< pno t and t -< P y W s. Now, the only state 

^-a-priori bisimilar to x is, here again, x itself. Hence, for s — > \i x and E = {x}, there exists no transition 
t — y V such that >l(E) < v(~^ no (E)) + -g, hence s and t are not ^-a-priori bisimilar. As a consequence, 

two-way a-priori simulation is different from a-priori bisimulation, and the negation is needed in the 
logical characterization of bisimilarity. 

Decidability of A Priori Simulation. An interesting fact is that it is NP-hard to decide a-priori 
simulation and bisimulation, even when e = 0. This contrasts with classical results on strong simulation 
and bisimulation whose decision procedures were proven to be in Poly-time (see El|24l|6l). The proof 
of the following theorem, not presented here due to lqck of space, is by reducing the subset sum problem, 
known to be NP-complete ([ 16]), to our problem. 

Theorem 4 The following problem is NP-complete: 
Input: APAy, s,t £ S. Question: Do we have s -< prio t? 

3.2 The logic Jf N for PAs 

We saw in the previous subsection that Jz? is not strong enough for PAs. We now give a logic character- 
izing our relaxed relations -< e and ~ e . The difference between this logic and jSf -1 is the modal operator 
that permits to "isolate" a distribution out of a state, and write properties that it satisfies. This allows the 
semantics to be defined on states, as pointed out as well by D'Argenio et al. [9]. In contrast, Parma and 
Segala rf20ll used a semantics on distributions to prove the logical characterisation of bisimulation (with 
£ = 0). 

Definition 8 (The logic jSf '""') The syntax differs by one operator from Jz?^: 

££ n ^ := T| --0 I 0i A0 2 | 0i V 2 I (a) {{fr, Pi )} ieI I finite, Pi G Qn [0; 1]. 
We write J£ for the same logic without negation. Let £ £ [0; 1]. The relaxed semantics |= £ of Jf N '^ 
is defined by structural induction on the formulas, in the same way as for 5£^ except for the modal 
formula: s |= e (a) {((j>i,Pi)}iei iff there exists a transition s —> \ifrom s such that for all i S /, we have 

<l{M E )>Pi-£. 

As for the logic jSf -1 we can, by structural induction on the formulas, construct for each <p S Jj? N '~^ 
an associated formula </> e G Jzf^ such that [</>] e = [0 e ]. Hence, here again, model checking of formulas 
with the relaxed semantics can be done using the same technique as for the usual semantics. The logics 
S£ N and £? N >^ induce the relations -<f and ~jF as in Def. km The following example illustrates 
how this logic differs from J/f '. The key difference is that formula (a) {(<pi,Pi)}iei is not equivalent to 
\ei{a)p,^>i. 

Example 4 Consider the PAs of Ex. pi As s -<P'"'° t, we also have s -<g t by Theoreml$\ and hence every 
formula of ' J2? satisfied by s is also satisfied by t. However, s -fit and the formula 

(«>{((*i>iT,|),((fc)iT,i),((fc3>iT,^)} 

of y N is not satisfied by t. Note how the semantics forces the Vj 's to commit before the three "bi 
formulas" are checked. 
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The following theorem is a logical characterization of the £ -relations. Notice that we need negation 
once again, since two-way simulation is different from bisimulation. 

Theorem 5 -< e =-<f and~ e =r^f ' . 

Proof. [-< e C-<^ ]. Let R be an £-simulation. We prove by structural induction that for all G Jf N , 
^(M) ^ Me- We prove the case where = (a) {(<pi,Pi)}iei, since the other cases are trivial. Let 
s G [0], t G /?({*}) and let s — > [l be the associated transition such that for all / G /, ju([0i]) > Pi- Since 
R is an e-simulation, there exists a transition t — > v such that /xJz? e (,R)v. Thus, for all E C 5, we have 
M(£) < v(/?(£)) + £. In particular, given z G /, we get that jU(M) < v(/?([0,-])) + £. But by induction 
hypothesis, we know that for all i G I, /?([0,-]) C [0,] e . This gives us that ju([[0,-]) < v([0,-] e )) + £ hence 
the result since by hypothesis ju([0i]) > Pi- 

\-< e 'D-<f' ]. We prove that -<f is an £-simulation. Suppose that s -<f t, and let s A fX be a transition 
from s. We need to find some t — > v such that v(-<f (X)) > /x(X) - £ for all X C 5. This will be 
constructed from a family of ? — >• Ve for finite sets £\ Let «GN and let £" C S be a finite set such that 
H(E)>n(S)-l/n. 

The key idea is to define the formula 0* = A^u^ j<ktyj f° r every e G E, and where (<pj)j e ^ is an 
enumeration of the formulas of «Sf. Then for every finite set X C 5, we set 0| = V ee x0e > and we let 
p* := MIV^]). Since e |= 0*, we have [0*] D^f ({*}), 

p| = M([V eeX 0*]) > MHf (*)) > |i(X), 

and s |= (a) {(V ee x0*,/4)}xc£ for all k > 1. By hypothesis, we get that f |= e (a) {(V ee x0*,/4)}xcE 
for all A: > 1. Let v| be the associated transition. Then for all X C £" and for all A G N: 

vi([V ee x0 e *l)>/4-£>M(X)-£- 

Now, [Ve e x0ele is decreasing to -^jp (X) as A goes to infinity. Since the systems we consider 
are finitely branching, we can define a transition t — > Ve such that Ve is the limit of a subsequence of 
{v E }keN- That is, there exists an increasing function y : N — > N such that for all set Y C S we have 
lim^ooV^ ^(y) = Ve(Y). This implies that: Vs{-<f (X)) > /x(X) — £. We have proven the following: 
for all s A /x, for all £" C 5 finite, there exists (Avg such that for any XC£we have Ve{<^ (X)) > 
H(X) — £. Let Ek, k G N be a growing sequence of finite subsets of S such that S = U^er^t- Again, since 
the system is finitely branching, let v be the limit of a subsequence of {VE t }keN- As before, we get that 
for any X C S finite, v(-<f (X)) >n(X)-e. 






=~ e j It can be proven that ~ g " is an £-bisimulation by following the proof above and using the 
fact that ~ g "■ is a symmetric relation (which comes from the presence of negation in &£;). □ 

4 A Bisimulation Pseudo-Metric between PAs 

4.1 The pseudo-metric d 

The notion of £-bisimulation induces a pseudo-metric on states of a PA, given by the smallest £ such that 
the states are £-bisimilar. 

Definition 9 (Bisimulation metric) Given s,t G S, let d(s,t) = inf{£ | s ~ e t}. 
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Using the finite branching of our systems, we can prove d is a bisimulation pseudo-metric, i.e., states 
at distance zero are bisimilar. We now discuss the computation of this distance between all states of a 
given PA. We propose three approaches, the first being exact, and the others approximate. The two first 
compute the distance iteratively, updating a function <i, : S x S — ^ [0; 1], in the same way as it was done 
for deterministic PAs in lfT2l . This approach is close to the classical iterative algorithms for computing 
simulation and bisimulation on probabilistic systems, see [2, 6]. It makes use of a network flow compu- 
tation. The algorithm for the first approach is the left one in Fig. [21 Given d : 5 x 5 — > [0; 1], and e > 0, 
let R d e be the relation on S x S defined as: Rf (s,t) iff d(s,t) < e. 



Algorithm srf : exact computation 
Input: A finite PA & = (S,Act,S>). 
Output: d:SxS-*[0;l]. 
Method: 

Let d (s,t) = Vs,t G S x S. Let j = 
Until dj = dj+i do begin: 



*/+i 



dj. 



For all (s, t ) G S x S do begin: 
For all a G Act do begin: 
For all s — ^ ii do begin: 
Let dj + \{s,t) be the smallest 

£ G [0; 1] s.t. ]f4v such that 

the maximum flow of network 

yK(ii,v,Ri J )^>>i(S)-e. 
end end end 



7 = 7 + 1 end return d 



;-!• 



Algorithm 8B : computation up to \jn 
Input: A finite PA & = (S,Act, 9), n € N. 
Output: d:Sx5^[0;lj. 
Method: 

Let d o (s,0 = \/s,t E S x 5. Let j = 
For m = ft — 1 to do begin: 
Until dj 

d j+ i = 



dj + i do begin: 



A,j. 

For all (s,t) s.t. d(s,t) = do begin: 

For all a G Ac? do begin: 

For all s — > ii do begin: 

If 3 t — V V s.t. the maximum 

flow of network <yT(jU, v,/? J ) 

is < ii(S) — me/n then let 

<f/ + i(v) = (m + l)e/«. 
end end end 

7 = 7 + 1 end end return dj-i. 



Figure 2: Computations of metric d on 5? 



Proposition 3 Algorithm si correctly outputs the distance d between all pairs of states in S. Moreover, 
algorithm s/ runs in time 0{\S\ 9 ■ \Act\ ■ I 2 ), where I is the maximal number of transitions with the same 
label issued from a single state. 

This algorithm is quite expensive, and hence we propose other approaches that approximate the 
distance. The first one is a variation of algorithm stf and is the right-hand algorithm of Fig. |2J Let \jn 
be the accuracy we are interested in, for n G N. The idea is to compute ra/«-bisimulation iteratively, 
for m decreasing from n — 1 to 0. These relations are decreasing as £-bisimilarity implies e'-bisimilarity 
for any e' > e. At each iteration of that loop, states whose distance have not been established yet have 
d- value 0. At step m < n, these states will be given distance (m + l)/n if they are not m/n-bisimilar. 
The relation consisting of states at zero distance will decrease at every j step. For every pair of states, the 
worst number of flow networks to be computed will be n (this happens if the states are bisimilar). Hence 
the algorithm runs in 0(|>S| 5 • n ■ \Act\ ■ I ). Of course, some values of m can be ignored and we can save 
some time. 

The last algorithm that we propose uses recent work of Zhang et al. [24], to update efficiently the 
flow computation in algorithm srf. The algorithm of Zhang et al. computes strong bisimularity on a 
probabilistic automaton in time 0(\S\ • m 2 ), where m is the total number of transitions. By a slight 
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generalization of this algorithm to our context, we can compute £-bisimularity on 3" in time 0(|5| ■ m 2 ), 
for any given e. Using a dichotomic approach, given two states s and t, we can compute d(s,t) up to an 
additive approximation factor 8 in time 0(\S\ • m 2 • log(S)), and thus we can compute d up to an additive 
approximation factor 8 in time 0(\S\ -m ■ log (8)). 



4.2 The decayed distance d x 

In the previous definitions, differences in the far future have as much importance as those in the near 
future. We can relax the impact of the future, or instead the impact of short term transitions, by using a 
decayed relaxation. Instead of a fixed relaxation of parameter e, we can ask for a relaxation that changes 
as we get further from the starting state. As we get deeper through the transitions, the parameter could 
get bigger, hence diminishing the importance of further differences, or symmetrically, we could make the 
parameter smaller. In order to leave this flexible, we will use a function X : [0; 1] —> [0; 1]. If x < X(x), 
the future will be neglected whereas x > X(x) will make the future more precise. If X is the identity, 
we get the previous notions. We will describe below a natural choice for X, but first, let us define the 
new semantics. We write X n for to the rc-th self composition of function X, n E N, and A is the identity 
function on [0; 1]. Also, S£^ will be the set of formulas of 5£ N of depth at most nel 

Definition 10 (The (e, X) -semantics) Let s E [— 1; 1], and X : [0; 1] — > [0; 1] The syntax is the one of 
J£ N . The semantics \=^ is defined similarly as for J£ N except for the modal operator. Given E Jzf w , 
let [0]£ = {s£ S\s |=g </>}. Given s E S, s |=g (a){(^i,pi)}iel iff there exists a transition s—^fl such that 

for all i E I, we have MlIriMfgO > Pi ~ £ - This semantics induces the relations < e " '' and ~ £ " ' as 
inDef% 

The relations -< e " ' and ~ e " ' are defined for formulas of a given maximal depth n, because in 
order to compute ^g on S x S for a given e E [0; 1], we may have to compute the |=jL/ e \ for all n E N. 

Most of the time, one wants to give less importance to the future. In these situations, the decay is 
called a discount and could be exponential, as in lfT0l[T4ll . In our case, this would correspond to asking 
that there is a constant < c < 1 such that 1 — X(e) =c(l — e), i.e. X(e) = 1 — c- (1 — e). 

The associated simulation and bisimulation are variations of Def . [2j 

Definition 11 (Order n (e, A)-bi/simulation) Given n EN, an order n (s,X) simulation on y is a de- 
creasing sequence of relations Ro,...,R n on S such that Rq = S x 5, and for all i E [l',n], whenever sR{t, 
ifs —7- n, then there exists t — > V such that pt ££ ^ e ' (/?;_i ) V. We write s -^ e ' n t if there exists an order n 
(s , X) -simulation R n on 3 such that sR n t, and we write s ~ e '" t if s -< e '" t and t -< e '" s. 

Proposition 4 LetnE N. Then <f n ^ = -<*•" and ~g" ' A = ~J»" 

Given s,t E S, we define d^(s,t) = min{A~ n (l) | s ~^- 1 „/n t}. We can compute the distance d x using 
the algorithm of Fig. [3] 

Proposition 5 Algorithm <€ of Fig. ^runs in time Od^l 5 • \Act\ ■ I 2 ). 
Proof. Direct, using 0(|5| 3 ) flow network computations. □ 
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Algorithm c €: Computation of the discounted metric dx on 5? 
Input: A finite PA & = (S,Act, fjiVeN. 
Output: d :SxS-+ [0;l]. 
Method: 

Let do(s,t) = 1 for all s,t eSx S. Let 8 = l/N. 

Let R Q = S x S. 

For n = 1 to N do begin: 

For all (s,t) £ S x S do begin: 

For all a G Act do begin: 

For all s — > il do begin: 
Let/?„ + i =R n . 

If there exists no transition t — > v such that the maximum 

flow of the network ^V(/j.,v,R n ) is greater than or equal 

to ii(S) — (l—n-S), then: let d(s,t) = l—n-S, and let 

R n+1 =R n+1 -{(s,t)}. 
end end end end 

return d. 

Figure 3: Computation of metric dx on 3? 



4.3 Comparison to other metrics on probabilistic systems 

During the past ten years, several metrics have been defined in the context of Probabilistic Automata 
or closely related models such as Labeled Markov Chains ifTTl [T31 [T3Tl . reactive probabilistic transition 
systems [5], Markov Decision Processes lHH[T3]|2ll]> or more general game processes [1]. Most of these 
metrics are variations of the metric of iPTD . In [|5l Th. 4.6], an equivalent metric is defined as a terminal 
coalgebra, using category theory. In J3J, the same authors give an algorithm to compute in polynomial 
time this metric, relying on linear programming computation for a transshipment optimization problem. 
This approach is for deterministic models and it is applied in lfl4l and related papers to compute metrics 
between Markov Decision Processes. Most of these algorithms introduce a decay factor to make the com- 
putation tractable. In [ 1 ] the authors consider metrics between systems which allow non determinism, 
but the complexity of the algorithms presented in [8] to compute the metrics is at best PSPACE. 

The main difference between our metric and those is that differences along paths are not accumulated 
in ours, even in the discounted metric: other metrics all involve comparing (among others) the proba- 
bility of paths, and this makes these metrics straightforwardly different from ours, as we never multiply 
probability values. In lfTTl[T4l . the metric can be computed using a familly of functional expressions 
,^ c from states to [0; 1] that play the same role as the quantitative formulas of [1]. Given s,t states of 
the system, the distance d c (s,t) is then defined as d c (s,t) = sup^ e jr c |/ c (s) — f°(t)\. This distance is 
incomparable with ours, as shows the following example. 

Example 5 Let < £ < 1, and consider the systems of Fig. [4] The distance between s and t is always 
greater with our distance, as d c (s,t) = c 2 • £ 2 < £ = d(s,t). On the contrary, for £ < 1/2, there is some 
c for which the distance between u and v is smaller with our distance. Indeed, d c (u,v) >=c 2 (£/2 — £ 2 ), 
whereas d(u,v) = £. Hence we obtain d(u,v) < d c (u,v) by taking c E [0; 1] such that c (3e/2 — £ 2 ) > £; 
for example, with £ = 1/4 and c 2 > 4/5. The example can be adapted for £ > 1/2. 
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Figure 4: Our metric differs from those based on paths. 



4.4 The Metric J on a Process Algebra 

We consider a process algebra on an extension of the model of PAs: the extension is that we distinguish 
between Input and Output action labels. As in lfl5l . we assume a set of underlying labels £, and suppose 
that the labels of the PAs belong to a set L = L!UL?, where L? = {a?|a G 1} andL! = {a!|aGl} are the 
sets of Input and Ouput labels respectivelly. Two PAs S?\ = (Si,Acti,3i\,s\) and 5^i = {Si,Act%^2-,s^) 
will synchronize on labels in Act\ n Ac?2- 

We only present the operators for non-deterministic choice and parallel composition, as the other 
operators can be taken as in |[T5ll . 

Non Deterministic Choice. Let =5*1,..., =5^ be PAs with respective state spaces Si,..., St and initial 
states s®,...,s®. Let {ai, ...,a/} C L, and for each a,- let {Hij\j G [1;«j]} be a finite family of distributions 
on {s®,...,sl}. We define: =5*" = (+) ' j=l {s -4 jUij,j G [l;n;]}{=5*i,...,=5*fc}, a PA whose state space is 
5" = {^} W,- e [i ; w 5,-, and initial state s. Transitions from s are all the s -4 ju,-j. There may be several 
transitions from s with the same label a,. =5*" accepts the input label a or outputs the label a, depending 
on a G L? or a G LI. 

Parallel Composition. Given PAs =5*) = (5i ,Act{, @i,s®) , i = 1 , 2, we define the parallel composition 
y f = =5*i 1 1=5*2- The synchronisation is on labels in Acti C\Act2- The state space of =5*" is S' = Si x 52, with 
initial state s /0 = (s^s®). The set of labels of =5*" is Ac?' = Act\ UAc?2- Given /x and v two distributions 



on disjoint sets S\ and 52, given X C 5i x 52, let /I (g) v(X) 
f G 52, we expect the following synchronized transitions: 



E(j,f)eX M ( J ) " v (0- Given states s G 5i and 



• Synchronization on input labels in Acti n Ac?2: Va G Ac?i nAc?2, if s — > jU and f -4 v, then there is 
a transition (j, £ ) — > ju ® V on =5". 

• Synchronization in the Output/Input: if 5 -4 /x and f — >• v, then there is a transition (s,f) -4 ju <8) v 
(and symmetrically) 

• Asynchronous evolution on labels in Acti \Act2'. given a GAc?i\Ac?2if s— > }l, then (s,t) -4 n®8 t , 
where 5 f is the Dirac distribution on t (symmetrically on 52). 

We prove that the distance d is non expansive with respect to the parallel operator: when composing 
two processes with a third one, the distance does not increase. Non expansiveness with respect to other 
operators is more common. 

Theorem 6 d(=5*i||=5*\ =5*211=5*0 < d^Sty. 

Proof. Let s, si , S2 be three states of y, =5*i and =5*2 respectivelly. Let e > 0, and suppose 51 ~ e 52- We 
want to prove that si x s, which is a state of =5*i | |=5*', is e-bisimilar to S2 x s, a state of S^%\ |=5". We prove 



by induction on n G N that if si 
0Gif w , and jG {1,2}: 



-> n e S2, then si x s ~" i 2 x s. We will use the following notations: Given 
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l^Jy.ny = {sj x s\sj G Sj, s G 5, and Sj X s |= 
Where the semantics is taken on the PA 5?j\ \<9". Let 

left ; - ([(/>] y ,.||^) = {v G S_/|3w G 5 5.?. m x v |= </>} 

Given v G S, let 

right ; .(v, l$jy jlly ) = {u£Si\uxv\= 0} 

The key case is when <p = (a) {(<pi,Pi)}iei G JSf , with depth «. Suppose si x s |= 0. Then there 
exists a transition si x s A /ii (g> v on 5?' such that for all i G /, (jUi &) v)(|0,-]) > /?,-. 

By hypothesis, si ~ e *2. Hence, there exists a transition 52 —> \ii such that /ii = Sf e (~ e )A t 2- 
We know that for all i G /, 

Mi®v([^])= £ v(v)- £ jui(v). 

V€lefti([0i]) HGright^v.M) 

Given v G left( [</>,■]) (hence v G 5), we know that il 2 (~" £ right! (v, [0,])) > jUi (rightj (v, {fa})) - e. 
Moreover, by induction hypothesis, ~" right^v, [0,]) C right 2 (v, [</>;] e . Indeed, 

~g righ^ (v, [0,-])n 52 =~'J {« G5 2 |3w' G Si 5.?. w ~g u and w' |= 0,-. 

We get the result since by induction hypothesis, if u ~g u' and s' G S, we have w x s' ~" «' x 5'. 
This implies that ^ 2 (right 2 (v, |fc] e )) > jUi (rightj (v, [0,-])) - e. Finally, 

L v(v)- £ M2(v)> £ v(v)- £ Mi(v)-e. 

veleft 2 (Me) »Gright 2 (v,[0,-] £ ) VSlefti ([</>;]) HGright, (v,[0,]) 

Hence /x 2 v([0,-] e ) > jUi <8> v([0,-]) - £. This proves that s 2 x s \= e <p. D 

5 Examples 

We build a benchmark set of deterministic PAs to compare distances. The processes are variations of 
a basic one from which we delete some transitions. The state space of the basic PA is a square grid of 
nxn. The set of actions is {a}. All the computations on the state indices are done modulo n: the grid is 
a torus. The a-transitions from state (i,j) are as follows: 



from (i,j) to (i,y — 1) : 0.1 (1,7 + 1) : 0.5 (i- l,y) : 0.25 (i + 1,j) :0.15 



This basic PA is compared to variations of it obtained by deleting in some states the transition of label 
a (to all successors). Note that the basic process is bisimilar to the one-state process that can do a with 
probability 1. We consider the distances between states with same indices of the different systems. Fig. [5] 
illustrates some PAs and the impact of the deletion of transitions for the two distances that we defined. 
The distance d^ is illustrated in the bottom grids of the figure. The linear function for X is the following, 
with N = 20. We take 8 = l/N, and let X(x) =x+ 8 if x G [0;1 - 8], and X{x) = 1 if jce [1 — 5; 1]. 
One can observe that the decay distances fade out when further from the difference, whereas d is more 
constant. 

It would be nice to compare these grids with others obtained from other known metrics. We leave 
that for future work, as we have no implementation of other metrics that can handle more than 25 states. 
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Figure 5: The red entries of the top grid are states that have lost their a-transition. The two other grids have at entry (i,j) the 
distance between state (/, j) of top grid to state (;', j) in the basic PA. The darker is the entry, the bigger the distance. The middle 
grid illustrates the basic distance, the other the decayed one. 



6 Conclusion 



We presented relaxed notions of e-simulation and £-bisimulation. When e = we retrieve the usual 
notions of bisimulation and simulation on PAs. We gave logical characterisations of these notions and 
algorithms to compute in PTIME two corresponding pseudo-metrics, one that discounts the future, and 
one that does not. We showed that our distance is not expansive with respect to process algebra oper- 
ators. We also showed that the basic logic Jzf ^ characterises a notion weaker than £-(bi)simulation, 
called a priori e-(bi)simulation. Interestingly, we have proven this notion NP-difficult to decide. Further 
work includes relaxing what is called probabilistic bisimulation and studying the associated distances; 
implementing our third proposal of algorithm to compute d, using a modification of the algorithm of 
[24]; investigating further the weaknesses and strengths of the different metrics defined so far. 
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